On Discriminative Joint Density Modeling
نویسندگان
چکیده
We study discriminative joint density models, that is, generative models for the joint density p(c,x) learned by maximizing a discriminative cost function, the conditional likelihood. We use the framework to derive generative models for generalized linear models, including logistic regression, linear discriminant analysis, and discriminative mixture of unigrams. The benefits of deriving the discriminative models from joint density models are that it is easy to extend the models and interpret the results, and missing data can be treated using justified standard methods.
منابع مشابه
A Generative Parser with a Discriminative Recognition Algorithm
Generative models defining joint distributions over parse trees and sentences are useful for parsing and language modeling, but impose restrictions on the scope of features and are often outperformed by discriminative models. We propose a framework for parsing and language modeling which marries a generative model with a discriminative recognition model in an encoder-decoder setting. We provide...
متن کاملDiscriminative Methods for Transliteration
We present two discriminative methods for name transliteration. The methods correspond to local and global modeling approaches in modeling structured output spaces. Both methods do not require alignment of names in different languages – their features are computed directly from the names themselves. We perform an experimental evaluation of the methods for name transliteration from three languag...
متن کاملRestricted Bayes Optimal Classifiers
We introduce the notion of restricted Bayes optimal classifiers. These classifiers attempt to combine the flexibility of the generative approach to classification with the high accuracy associated with discriminative learning. They first create a model of the joint distribution over class labels and features. Instead of choosing the decision boundary induced directly from the model, they restri...
متن کاملImplicit Modeling - A Generalization of Discriminative and Generative Approaches
We propose a new modeling approach that is a generalization of gen-erative and discriminative models. The core idea is to use an implicitparameterization of a joint probability distribution by specifying only theconditional distributions. The proposed scheme combines the advantagesof both worlds – it can use powerful complex discriminative models asits parts, having at the s...
متن کاملar X iv : 0 80 7 . 34 70 v 2 [ st at . M L ] 1 8 N ov 2 00 8 Inference with Discriminative Posterior
We study Bayesian discriminative inference given a model family p(c,x, θ) that is assumed to contain all our prior information but still known to be incorrect. This falls in between “standard” Bayesian generative modeling and Bayesian regression, where the margin p(x, θ) is known to be uninformative about p(c|x, θ). We give an axiomatic proof that discriminative posterior is consistent for cond...
متن کامل